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Abstract 

In this work we introduce the concept of s-sparse observabihty for large systems 
of ordinary differential equations. Let i = f{t,x) be such a system. At time 
T > 0, suppose we make a set of observations b = Ax{T) of the solution of the 
system with initial data x{0) — x'^, where A is a matrix satisfying the restricted 
isometry property. The aim of this paper is to give answers to the following 
questions: Given the observation 6, is x'^ uniquely determined knowing that 
is sufficiently sparse? Is there any way to reconstruct such a sparse initial data 
x°7 
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1. Introduction and Results 

In recent years a number of papers on signal processing have developed a 
series of ideas and techniques on the reconstruction of a finite signal x € 
from many fewer observations than traditionally believed necessary. It is now 
common knowledge that it is possible to exactly recover x knowing that it is 
sparse or nearly sparse in the sense that it has only a limited number of nonzero 
components. A more formal definition of sparsity can be given through the Iq 
norm ||a;||o '■— #{« : Xi ^ 0}, that is, the cardinality of x's support. If ||2:||o < s, 
for s a nonnegative integer, then we say that x is s-sparse. 

Since sparsity is a very often encountered feature in signal processing and 
many other mathematical models of real-life phenomena, estimation under spar- 
sity assumption has been a topic of increasing interest in the last decades. At 
this point, the work on this subject is so extended and growing so rapidly that it 
is extremely difficult to mention without injustice its achievements and results. 
It is beyond the scope of this paper to review, even partially, the contributions to 
this new and very dynamic area of research. For example, in the signal process- 
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ing case, the interested reader can find valuable insight in the very informative 
survey by Bruckstein, Donoho and Elad 

This work addresses the recovery of the initial state of a high-dimensional 
dynamic variable from a restricted set of measurements, knowing that the initial 
state is sparse. More precisely, let x{-) be the solution of the following initial- 
value problem 

x(t) = fit,x{t)), for t>0; x{0) ^ x° . (1) 
Suppose that we can observe 

b = Ax{T) (2) 

at a certain time T > 0, where the vector b represents the observations, and A 
is an n X m measurement matrix (dictionary). As in signal processing case, the 
more interesting situation is when n << m; one interprets b as low-dimensional 
observations/measurements at time T > of the high-dimensional dynamic 
solution x{-). Here are two interesting questions that we address in this note: 

Question 1: Given the observation 6, is x^ uniquely determined knowing 
that x'^ is sufficiently sparse? 

Question 2: Is there any way to reconstruct such a sparse initial data x"? 

Hereafter, / : [0, T] x M™ — ^ is a Lipschitz function in the second variable, 
i.e., there is L{-) e L°°(R™) such that \\f{t,x) - fit,y)\\ < Lit)\\x - y\\ in 
[0,r]xM". ByL wedenotetheL°°-normofL(-) over the interval [0,r]. For x e 
R™, the /P-norm {p > 1) of x is defined as usually \\x\\p := {J2Zi In 
what follows, we also assume that the matrix A satisfies the restricted isometry 
property. Let us recall the concept of restricted isometry constants (see Q)- 

Definition 1. For each integer s = 1, 2, . . . define the isometry constant of a 
matrix A as the smallest number such that {1 — Ss)\\x\\2 < ||^a;||2 < (l + (5s)||a:^||2 
holds for all x G R™ with \\x\\q < s. 

The following definition introduces a new concept, that is, the notion of 
s-sparse observability. 

Definition 2. The pair is called s-sparse observable at time T > if 

the knowledge of b allows us to compute the s-sparse initial data vector x'^ . 

In other words, (HJ-lIl]) is s-sparse observable at time T > if for any so- 
lutions xi{-) and X2{-) corresponding to the s-sparse intial data a;° and a;^, 
respectively, with Axi{T) = Ax2{T), we have x^ = 

Our first result gives a positive answer to Question 1. In fact it provides 
a sufficient condition for s-sparse observability. Roughly speaking, it says that 
s-sparse observability holds for sufficiently short periods of time. 

Theorem 1. LetT < ^ In (1 + ^^p|^)- Then dJ)-© is s-sparse observable at 
time T. 

In the remainder of this section we indicate how the sparsest initial data x^ 
can be found, or approximated. We consider the more applicable situation in 
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which the measurements at time T are corrupted with noise. That is, 

b = Ax{T) + e, (3) 

where e is the noise term whose maximum magnitude is e (i.e., ||e||2 < e). 

Suppose we seek the sparsest initial data a;° that solves ([T]) and ([3]) . In order 
to narrow down to one well-defined (sparse) solution, we consider the problem: 

(Pn) Find X :— ars min ||a;||o. 

||6-Aa;(T)||2<c 

Here x{t) is the solution of ([TJ together with the initial condition a:;(0) = x. 
However, this is a very hard combinatorial-dynamic optimization problem and 
practically impossible to solve. We propose reconstructing x° by replacing the 
lo norm |j • |jo with the weighted h norm || • (||a;||i,u, := YT=i Wi\xi\, Wi > 0, 
i — l,2,...,m), which is, in a natural way, its best convex approximant. This 
strategy originates in the work of Santosa and Symes ^ in the mid-eighties (see 
also for early results). In this context, we seek x'^ as the solution to the 

dynamic optimization problem 

(Pl^) Find := arg min ||a;||i,u,. 

Denote by r the condition number of the matrix W :— diagjwi}, that is, r := 
maxi<2<77j {wi}/ mmi<i<m{wi} , and by Xs the vector having only the s largest 
entries of the vector x, the others being set to zero. The following result gives 
a positive answer to Question 2. In essence, it states that for sufficiently small 
times, the accurate recovery of the sparse initial data cc" can be done. Its proof 
is largely influenced by the methods and techniques used in 0, Q . 

Theorem 2. // ^2. < (1 + T^/2)-l and T < ^ In (l + ' ^^""^ 

the solution x* to {P^^^) satisfies \\x* - x'^\\2 < Cos^^/'^\\x° - a;°||i + Cie, with 
Co and Ci constants independent of x'^ . 

2. Proofs of Results 

2.1. Proof of Theorem]^ 

Suppose that ([ij-lll]) is not s-sparse observable at time T. That is, there 
exist s-sparse vectors xl and Xj, with xl ^ X2, such that Axi{T) ~ Ax2{T), 
where xi{t) and X2{t) are solutions to ^ together with the initial data a;i(0) = 
x\ and X2{Q) = X2, respectively. Since xi{t) ^ x\ + J^^ f{s,xi{s))ds and 

X2{t) = x'^ + f{s,X2{s))ds, it follows that \\x2{t) ~ Xi{t)\\2 < \\x'^ - X^2 + 

/o \\f{s,X2{s)) - /(s,xi(s))||2ds < \\x^2 " 4h + Jo L{^)\\^2{s) - x^{s)\\2ds. By 
Gronwall's inequality, one obtains ||a;2(i) — a;i(t)||2 < \\x2 — Xi\\2e-l'o ■f'(«)<i«^ and 
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so 

= \\AX2{T) - Ati(T)||2 = \\A{xl ~ X°,) + A [ [/(<,X2W) - /(t,.Ti(i))]dt||2 

Jo 

> \\A{X° - X°)\\2 ~ \\A\\ [ m\\x2{t) - Xi{t)\\2dt 

Jo 

> - x°,)\\2 - II All r LW||a;° - a;°||2e/o "^^'^"'dt 

Jo 

>\\Aix°2~4)h-M\\A\\-\\x°2~4h, 

where M e^o L(s)ds „ r^j^^g^ p^^o _ ^o^n^ < ^^||^|| . ||^o _ 2;0||2. Then, 
because a;^ — is 2s-sparse and from the restricted isometry property, we obtain 
that VI - < M\\A\\, which cannot hold for T < iln(l + ^^^^)- 

2.2. Proof of Theorem\E 

The following Lemma is a simple application of the parallelogram identity 
and is due to Candes 0, Lemma 2.1.]. We include it here, together with its 
proof, for reader's convenience. 

Lemma 3. Let x and x' he two vectors in R™. Suppose that x and x' are 
supported on disjoint subsets and \\x\\o < s and \\x'\\q < s'. Then \{Ax, Ax') \ < 

Ss+s'M2\\x'\\2. 

Proof. Denote by y — a::/||a;||2 and y' — a;'/||x'||2 the unit vectors in the x and x' 
directions, respectively. By the restricted isometry property, it is easy to see that 
2(1 — Ss+s') < \\Ay±Ay'\\l < 2(1 + Sg+s'). These inequalities, together with the 
parallelogram identity, give \{Ay,Ay')\ = l\\\Ay + Ay'\\l-\\Ay-Ay'f2\ < 5,+^', 
and so | (Aa;, Aa;') | < (Ss+s'||2^ll2||2;'||2, which concludes the proof. □ 

Let y(t) be the solution to the initial value problem y{t) = f{t,y) in (0,r), 
y(0) = a;*, where x* is the solution to (Pl^^). Then, 

\\Ay{T) - Ax{T)\\2 < ||6 - Ay(T)||2 + ||& - Aa;(r)||2 < 2e. (4) 

Decompose X cLS X — X + h. Then, as in [2|, decompose h into a sum of vectors 
/iTo, hTi, each of s-sparsity. Here, To corresponds to the locations of the first 
s largest coefficients of h, Ti to the locations of the next s largest coefficients, 
and so on. Since x{t) ~ x'^ + f{s,x{s))ds and y{t) — x* + f{s,y{s))ds, 

it follows that \\y{t) - x{t)\\2 < \\h\\2 + L{s)\\y{s) - a;(s)||ds. By Gronwall's 
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inequality, we get \\y{t) ~ x{t)\\2 < Wh^efo ^^'''>'^'' , for all t £ [0,T]. Then, 

\\Ay{T) - Ax{T)h = \\Ah + A f [f{t, y{t)) - f{t, xmdth 

Jo 

> ||A/^||2- ||A|| / L{t)\\y{t) ~ x{t)\\2dt 

Jo 

>\\Ah\\2-\\A\\ [ L(i)||/i||2e/oi'(-)'i-dt 
Jo 

> ||A/i||2-M||A||||/i||2, (5) 
where M := e^o i^^")'^" - 1. From g]) and ([5]), wc obtain 

||A/i||2 < 2e + MP||||;i||2. (6) 

Next, we estimate |1/i(ToUTi)<=I|2, where /i(j.^y7'j)c := h — hT„ — ft-Ti- First off, 
observe that ||/it,||2 < s^^^\\hT,\\oo < s'^/^HH-i 111' j ^ 2, and so 

J2\\hT,h<s-'/'J2\\''T,\\i=s-'/'\\hTs\\i, (7) 
where hr^ := h — hT„- Thus, 

II VoUT,)^l|2 = II J2 - ^ - ""'^'lIHHIl- (8) 

i>2 i>2 

Because x° is feasible, it satisfies ||a;*||i.u, < |la;°|li,uj, which implies 

m rn 

X!^»|2^*l > Xl^^'l"^^ + ^^'1 = X!^*l^» +'^'■1+ I] u'»|a;° + /ii| 

i=l i=l ieTo iGTo'^ 

ieTo ieTo ieTg ieTg 

and so ||/iT(j=||i,tu < ||/iTnl|i,to + Sjlx" — x2||i,to. This last inequality induces 
mini<j<„{it;j}||/iT-||i < maxi<i<™{wi}(||/iTol|i + 2||a;° - x°\\i), and so 

\\hTs\\i<r{\\hTji+2\\x°-x%). (9) 

From ([S]), (|n]), and the Cauchy-Schwarz inequality, it follows that 

WhTounrh < rs-'/^i\\hT,\\i + 2\\x" - x% < T{\\hT„h + 2eo), (10) 
with eo := s^^/^\\x^ — x^\\i. Now, let us estimate ||/itouTi lU- By the restricted 
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isometry property, it follows that 
1 



/ I I X II^^ToUTilh < \\hT0UTA\2 < . II^^ToUTilb- (11) 

V 1 + C'2s V 1 - 02s 

Since A/itqUTi = Ah- Y.j>2 ^^t^ , we have that || A/itquTi |! 1 = (^/itouTi , Ah) - 
Y.3>2^AhT,,uT^,AhT^) < {{AhTouT^,Ah)\+J2J>2\{AhTouT^,AhT^)\. This and 
the restricted isometry property imply that 

\{AhTouT^,Ah)\ < \\AhT„uTA\2\\Ah\\2 < p/iToUTj|2(2e + A/||A||||/^||2) (by ®) 



< yT+^||/iT„uTj|2(2e + M||A||||/i||2) (byCJ)). (12) 
From Lemma [21 we have that \{AhTi, Ahxj)] < <52s||^Ti Ibll/iTj II2, for « ^ j, and 

so \{AhToUT,,AhT,)\ < <52.(||/lTo||2 + ||/iTi||2)||/itJ|2 < 72^2^ || /iToUtJ! 2 1| /^T, ||2- 

Therefore, 



||2(2e+M||A||||/l||2)+\/2,52.||/lToUTj|2^|IH||2. 

As in y , let a = 2y/TT67s{l - <52s)"^ and p = V262s{l - 62^)^^- Then, 
II/1T0UTJI2 <a(e+^MP||||/i||2)+p^||/iT,||2 (by m and (0) 

J>2 

<a{e+hd\\A\\\\h\\2)+ps-'/^hTs\\i (by ©) 

< a(e+ iMP||||/i||2) + ps-'/M\\hTo\\i + 2|lx° - x%) (by ©) 

< a(e+ iM|lA|l|l/i|l2) +pr|l/iTouTi||2 + 2preo, 

and so 

\\hT0UTA\2 < (1 - P^)"M«(e + ^AfP||||;i||2) + 2preo]. (14) 

(Observe that 1 - pr > since (52s < (1 + \/2r)"^) 
Then, 

||/^||2 < ||/lToUTi||2 + ||/l(ToUTi)=l|2 < (1 + t) || /iToUTi || 2 + 2x60 (by (HHD) 

< {1 + t){1- pT)-Aaie+^M\\A\\\\h\\2) + 2pTeo]+2Teo, (by dH) 

and so \\h\\2 < CoCq + Cie, with Co := 2t{p + 1)[1 - pr - 0.5a{l + t)M\\A\\]-^ 
and Ci := «(! + t)[1 - pr - 0.5q:(1 + t)M||A||]"^ As a consequence of T < 

In ^1 + (■]^_^^^^||_^"'||^^^^^ ^ , observe that both Co and Ci are strictly positive. 
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